1. Field of the Invention
The present invention relates to an image processing apparatus and image processing method that capture an object that is larger than a view of an image capturing apparatus by a plurality of times to obtain a captured image of the entire object.
2. Description of the Related Art
An electric panhead has been conventionally known as a device that is attached between a tripod and a camera and used to electromotively rotate the camera, which is set in its upper portion, through a predetermined angle. In recent years, a system which captures a plurality of images while changing the orientation of the image capturing camera by driving the electric panhead, and connects the captured images to each other, thereby generating an image (for example, a panoramic image) with a very large number of pixels is attracting a great deal of attention. A most serious challenge in such an image capturing system is an image connection process of the captured images. In a technique known as the image connection process, adjacent images are estimated in accordance with the degree of similarity between their edges, and are connected together. However, if the object is captured as images that have uniform regions or are partially even and therefore have little difference between them, it is very difficult to correctly specify adjacent images such that a satisfactory connection result can not be obtained with this technique. Also, because different errors are generated in respective captured images due to the influence of, for example, the lens aberration and the object perspective, if a simple comparison is insufficient to obtain correct corresponding points, no appropriate connection result may be obtained either.
Under the circumstance, to calculate the correct corresponding points, a technique to perform the following coordinate transformation, for example, has been proposed (see, for example, Japanese Patent Laid-Open No. 07-174538). According to this technique, first, based on a plurality of image data, the relative rotation angle between an object and an input means, and the position of the rotation axis of an image capturing apparatus, are calculated as coordinate transformation parameters used to connect the plurality of image data. Next, based on the calculated coordinate transformation parameters, the target images are coordinate-transformed to those in one coordinate system and composited.
To prevent the influence of the difference in luminance for each image data upon composition, a method of comparing the second-order derivative of each pixel has also been proposed, instead of simply calculating the differences between these image data in portions in which they overlap each other, in order to obtain corresponding points.
However, according to the coordinate transformation technique disclosed in Japanese Patent Laid-Open No. 07-174538, corresponding points in regions in which adjacent images overlap each other are obtained, and a process of connecting these images is performed. For this reason, images including fine repetitive patterns, such as a mounting of a hanging scroll, may not be able to be correctly connected to each other. To cope with this situation, images can be deformed/connected without obtaining corresponding points. Nevertheless, in this case, so-called lens aberration correction must be adopted, where the lens distortion, the vignetting, and the magnification chromatic aberration, for example, are corrected. An error of the captured image, which is generated by the lens aberration, variously changes depending on the object distance (shooting distance). Hence, a method of calculating the relative position between images, as in Japanese Patent Laid-Open No. 07-174538 mentioned above, is insufficient to appropriately correct this error, so the object distance for each image or the camera position in image capture must be correctly estimated.
Also, appropriate perspective correction cannot be performed by simply adopting coordinate transformation such as rotation/translation for the captured images. For appropriate perspective correction, again, the camera position in image capture must be appropriately estimated.
Furthermore, when vignetting correction is adopted in accordance with camera settings in image capture, the amount of noise increases in a direction away from the image center. Upon adopting such correction, even the method of using a second-order derivative, mentioned above, is insufficient to eliminate the influence of the luminance.